The present invention relates to audio signal decoding and audio signal processing, and, in particular, to a decoder and methods for adapting audio information in spatial-audio-object-coding (SAOC).
In modern digital audio systems, it is a major trend to allow for audio-object related modifications of the transmitted content on the receiver side. These modifications include gain modifications of selected parts of the audio signal and/or spatial re-positioning of dedicated audio objects in case of multi-channel playback via spatially distributed speakers. This may be achieved by individually delivering different parts of the audio content to the different speakers.
In other words, in the art of audio processing, audio transmission, and audio storage, there is an increasing desire to allow for user interaction on object-oriented audio content playback and also a demand to utilize the extended possibilities of multi-channel playback to individually render audio contents or parts thereof in order to improve the hearing impression. By this, the usage of multi-channel audio content brings along significant improvements for the user. For example, a three-dimensional hearing impression can be obtained, which brings along an improved user satisfaction in entertainment applications. However, multi-channel audio content is also useful in professional environments, for example, in telephone conferencing applications, because the talker intelligibility can be improved by using a multi-channel audio playback. Another possible application is to offer to a listener of a musical piece to individually adjust playback level and/or spatial position of different parts (also termed as “audio objects”) or tracks, such as a vocal part or different instruments. The user may perform such an adjustment for reasons of personal taste, for easier transcribing one or more part(s) from the musical piece, educational purposes, karaoke, rehearsal, etc.
The straightforward discrete transmission of all digital multi-channel or multi-object audio content, e.g., in the form of pulse code modulation (PCM) data or even compressed audio formats, demands very high bitrates. However, it is also desirable to transmit and store audio data in a bitrate efficient way. Therefore, one is willing to accept a reasonable tradeoff between audio quality and bitrate requirements in order to avoid an excessive resource load caused by multi-channel/multi-object applications.
Recently, in the field of audio coding, parametric techniques for the bitrate-efficient transmission/storage of multi-channel/multi-object audio signals have been introduced by, e.g., the Moving Picture Experts Group (MPEG) and others. One example is MPEG Surround (MPS) as a channel oriented approach [MPS, BCC], or MPEG Spatial Audio Object Coding (SAOC) as an object oriented approach [JSC, SAOC, SAOC1, SAOC2]. Another object-oriented approach is termed as “informed source separation” [ISS1, ISS2, ISS3, ISS4, ISS5, ISS6]. These techniques aim at reconstructing a desired output audio scene or a desired audio source object on the basis of a downmix of channels/objects and additional side information describing the transmitted/stored audio scene and/or the audio source objects in the audio scene.
The estimation and the application of channel/object related side information in such systems is done in a time-frequency selective manner. Therefore, such systems employ time-frequency transforms such as the Discrete Fourier Transform (DFT), the Short Time Fourier Transform (STFT) or filter banks like Quadrature Mirror Filter (QMF) banks, etc. The basic principle of such systems is depicted in FIG. 3, using the example of MPEG SAOC.
In case of the STFT, the temporal dimension is represented by the time-block number and the spectral dimension is captured by the spectral coefficient (“bin”) number. In case of QMF, the temporal dimension is represented by the time-slot number and the spectral dimension is captured by the sub-band number. If the spectral resolution of the QMF is improved by subsequent application of a second filter stage, the entire filter bank is termed hybrid QMF and the fine resolution sub-bands are termed hybrid sub-bands.
As already mentioned above, in SAOC the general processing is carried out in a time-frequency selective way and can be described as follows within each frequency band, as depicted in FIG. 3:                N input audio object signals s1 . . . sN are mixed down to P channels x1 . . . xP as part of the encoder processing using a downmix matrix consisting of the elements d1,1 . . . dN,P. In addition, the encoder extracts side information describing the characteristics of the input audio objects (side-information-estimator (SIE) module). For MPEG SAOC, the relations of the object powers w.r.t. each other are the most basic form of such a side information.        Downmix signal(s) and side information are transmitted/stored. To this end, the downmix audio signal(s) may be compressed, e.g., using well-known perceptual audio coders such MPEG-1/2 Layer II or III (aka .mp3), MPEG-2/4 Advanced Audio Coding (AAC) etc.        On the receiving end, the decoder conceptually tries to restore the original object signals (“object separation”) from the (decoded) downmix signals using the transmitted side information. These approximated object signals ŝ1 . . . ŝN are then mixed into a target scene represented by M audio output channels ŷ1 . . . ŷM using a rendering matrix described by the coefficients r1,1 . . . rN,M in FIG. 3. The desired target scene may be, in the extreme case, the rendering of only one source signal out of the mixture (source separation scenario), but also any other arbitrary acoustic scene consisting of the objects transmitted. For example, the output can be a single-channel, a 2-channel stereo or 5.1 multi-channel target scene.        
FIG. 6 schematically depicts the principle of an audio encoding/decoding scheme. In particular, FIG. 6 is a principle description of an audio encoding/decoding chain.
At the encoding side, the audio signal is compressed by an audio coding scheme (typically exploiting perceptual effects) and Parametric Side Information (PSI) is computed (see encoder 601). The resulting bitstream consisting of coded audio signal and PSI are stored (or transmitted) to the decoder side, where they can be decoded by various decoder instances 620, 621, 622, labeled as “A”, “B”, etc. in FIG. 6. These decoder instances can differ from each other (e.g., different complexity levels in standard specification, application or implementation restrictions, etc.) [SAOC, SAOC1, SAOC2].
State of the art coding schemes are not capable to adapt the PSI to a specific target application scenario or platform in an efficient way. This can lead into higher (than necessitated) computational complexity at the decoder side or can result in compatibility problems.